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Abstract: 

PROBLEM TO BE SOLVED: To provide a moving picture encoder capable of enhancing view quality of a moving picture so as to smoothly reproduce a stable 
moving picture. SOLUTION: An encoding mode setting section 6 estimates a generated code amount when one picture is intra-coded and also estimates a 
buffer occupied amount of a virtual buffer when the generated code amount is removed. The encoding mode setting section 6 estimates a frame rate on the 
basis of the estimated buffer occupancy amount. A device control section 7 designates a desired frame rate. The moving picture encoder decides whether or 
not a difference between the estimated frame rate and the designated frame rate is a prescribed or more and executes the intra-coding when the decided 
difference of the frame rate is less than the prescribed value or executes the inter-coding when the decided difference of the frame rate is the prescribed 
value or more, as its control. 
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(57) Abstract 

SUBJECT The quality of the appearance of video is raised and that the smoothly stable video is reproduced aims at providing the video coding equipment 
which becomes possible. 

Means for SolutionThe coding mode set part 6 presumes a generated code amount when one picture is formed into a screen inner code. Buffer occupied 
quantity of a virtual buffer when this generated code amount is removed is also presumed. The coding mode set part 6 is based in this presumed buffer 
occupied quantity, and presumes a frame rate. The device control part 7 specifies a desired frame rate. It is judged whether a difference of a presumed frame 
rate and said frame rate specified is beyond a predetermined value. It is based on video coding equipment controlled to perform screen inner code-ization 
when a difference of a judged frame rate is less than a predetermined value, and to perform coding by the coding between screens when a difference of said 
judged frame rate is beyond a predetermined value. 



Claim(s) 

Claim IVideo coding equipment which performs coding while referring to a virtual buffer characterized by comprising the following for guessing an occupation 
of a buffer by the side of decoding. 

A generated code amount estimation means which presumes a generated code amount when one picture is formed into a screen inner code. 

A buffer-occupied-quantity estimation means which presumes buffer occupied quantity of said virtual buffer when a generated code amount presumed for 

said one picture is removed. 

A frame rate estimation means which is based in said presumed buffer occupied quantity, and presumes a frame rate. 
A frame rate setting means which specifies a desired frame rate. 
Said presumed frame rate. 

A frame rate judging means which judges whether a difference with said frame rate specified is beyond a predetermined value. 

An encoding control means to control to perform screen inner code-ization when a difference of said judged frame rate is less than a predetermined value, 
and to perform coding by the coding between screens when a difference of said judged frame rate is beyond a predetermined value. 

Claim 2By measuring a setting-out means to set a threshold as a virtual buffer, and buffer occupied quantity of said virtual buffer immediately after coding 
the threshold concerned and one picture, The video coding equipment according to claim 1 providing further a frame rate update means which updates a 
frame rate. 

Claim 3The video coding equipment according to claim 2, wherein said setting-out means sets two or more thresholds as buffer occupied quantity. 
Claim 4Said frame rate update means are two or more fields divided by threshold which has the lowest value from a threshold which has the largest value 
among said two or more thresholds, The video coding equipment according to claim 3 updating so that it belongs to a field where buffer occupied quantity of 
said virtual buffer immediately after coding one picture is low, and a frame rate may become low in order. 

Claim 5A frame number calculating means which computes the number of frames formed into the screen inner code by within a time with the past , The 
video coding equipment according to any one of claims 1 to 4 providing further a control means controlled to perform coding between screens when said 
frame number is more than a predetermined number, a frame number judging means which judges whether said computed frame number is less than a 
predetermined number, and. 



Detailed Description of the Invention 
0001 

Field of the InventionThis invention relates to the video coding equipment for elongating the video compressed especially about the video coding 

equipment for compressing video or elongating the compressed video, and outputting a clear picture. 

0002 

Description of the Prior ArtWith the video coding equipment in this invention. ITU-T (International.) H.26x and ISO/IEC (International Standards 
Organization/International.) of Telecommunication Union-Telecommunication sector advice It is a device for coding video with the video coding mode 
represented by standard MPEG (Moving Picture Experts Group) for Electrotechnical Commission. That is, video coding equipment performs coding per frame 
using a motion compensation, orthogonal transformation (for example, discrete cosine transform), etc. 

0003Generally, to the video signal inputted, the video coding mode represented by MPEG H.26x of an ITU-T recommendation and standard for ISO/IEC is 
spatial, and applies compression with temporal correlation. Based on the data obtained by this compression, according to the given order, variable length 
coding is performed further, and a code sequence (bit stream) is generated. 

0004Video coding equipment must output the bit stream of the code amount specified according to the predetermined encoding parameter. Furthermore, in 
the buffer by the side of a decoder, a generated code amount must be controlled by the encoder side supposing the occupation of the buffer by the side of a 
decoder so that neither overflow nor underflow arises. This buffer is called a VBV (Video Buffering Verifier) buffer. It may only be called a virtual buffer. As for 
the capacity of the VBV buffer, upper limit was decided by MPEG-4 with the profile and the level. A generated code amount is controlled by the quantizing 
scale used in order to quantize the discrete cosine transform (OCT: Discrete Cosine Transform) coefficient obtained as a result of DCTs being performed by 
every macro block (MB:MacroBlock) in a frame. Generally, a generated code amount and a quantizing scale have a relation of reverse proportion. It is 
possible to change a generated code amount free using this character. 

0005However, since a quantizing scale generally has restriction, it is impossible to control a generated code amount only by a quantizing scale. Then, when 
there are more generated code amounts than a desired value, the number of frame skips is increased. By increasing the number of frame skips, the frame 
which should be coded can be delayed and the underflow of a VBV buffer can be prevented. Stuffing is performed when there are few generated code 
amounts than a desired value. Stuffing is inserting a redundant bit and prevents overflow of a VBV buffer. 

0006Drawing 1 is a figure showing the conventional method of controlling a virtual buffer, and shows the buffer occupied quantity of the virtual buffer to 
time. As the control method of a frame skip, the boundary value (B (min)) in a VBV buffer is established. The amount of buffers is made to increase by 
skipping until it increases the number of frame skips when less than this (time t 4 shown in drawing 1), and it exceeds a boundary value. 
0007 

Problem(s) to be Solved by the InventionHowever, according to this method, when a state with many generated code amounts continues, a state with 
little buffer occupied quantity continues, and buffer occupied quantity moves the boundary-value circumference up and down. For example, it moves up and 
down like time t 4 shown in drawing 1, and time t 5 . Therefore, the operation which increases the number of frame skips and recovers a buffer, and operation 
of a predetermined frame skip are repeated. As a result, it will be in a state with an unstable frame interval, and quality will worsen dramatically also at 
appearance. 

0008In consideration of improvement in image quality, error tolerance, and random access nature, I-VOP (I-VOP: Intra -Video Object Plane) may be inserted 
into a bit stream. Since I-VOP has comparatively many code amounts, a VBV buffer will become an underflow tendency if I-VOP is inserted frequently. There 
is a problem that the number of frame skips increases rapidly, by inserting I-VOP around said boundary value. 

0009Then, the video where the purpose of this invention raised the quality of the appearance of video and which was stabilized smoothly being reproduced in 

view of the problem in these former is providing the video coding equipment which becomes possible. 

0010 

Means for Solving the Problemln video coding equipment which according to this invention performs coding while referring to a virtual buffer for guessing 
an occupation of a buffer by the side of decoding, A generated code amount estimation means which presumes a generated code amount when one picture is 
formed into a screen inner code, A buffer-occupied-quantity estimation means which presumes buffer occupied quantity of said virtual buffer when a 
generated code amount presumed for said one picture is removed, A frame rate estimation means which is based in said presumed buffer occupied quantity, 
and presumes a frame rate, A frame rate setting means which specifies a desired frame rate, and said presumed frame rate, A frame rate judging means 
which judges whether a difference with said frame rate specified is beyond a predetermined value, An encoding control means to control to perform screen 
inner code-ization when a difference of said judged frame rate is less than a predetermined value, and to perform coding by the coding between screens 
when a difference of said judged frame rate is beyond a predetermined value, It is provided by providing video coding equipment. 

OOllThis video coding equipment predicts a virtual buffer after I-VOP coding, and when changing sharply a frame rate obtained from that predicted value 
from the present frame rate, it does not code I-VOP. 

0012Two or more control points are formed in a virtual buffer, and a frame rate updating section which sets up a different frame rate for every point of the is 



mhtml :http ://ipportal .thomnet.com/patent/home/Patent Families Documents/PU/PU04xxx/. . . 2/14/2011 



rf PatentOrder MT 



Page 3 of 9 



provided. This frame rate set up every two or more points is sent to a coding part of video, and a coding part codes by skipping a picture inputted according 
to a specified frame rate. 

0013The number of I-VOP which should be contained within predetermined time from buffer occupied quantity of a virtual buffer is controlled, and buffer 

underflow is controlled. 

0014 

Embodiment of the InventionHereafter, with reference to Drawings, the embodiment of the video coding equipment of this invention is described. 
(A 1st embodiment) Drawing 2 is a functional block diagram showing the electric internal configuration of the video coding equipment in a 1st embodiment. 
The video coding equipment of this invention consists of the coding part 1, the output buffer 2, the virtual buffer 3, the frame rate updating section 4, the 
generated code amount control section 5, the coding mode set part 6, and the device control part 7. 

OOlSThe coding part 1 inputs the picture signal which should be coded from a picture input device. A picture input device is a signal source of a dynamic 
image signal like a digital video camera or recording playback apparatus. The inputted picture signal is coded according to predetermined video compression 
technology, i.e., MPEG-4 standard method for ISO/IEC and H.263 method of an ITU-T recommendation, or the method which changed these. Coding divides 
each frame contained in the inputted picture signal into a predetermined macro block according to video compression technology. From the coding mode set 
part 6, the coding mode (intra coding or inter encoding) which codes each macro block is inputted. 

0016Below, the video coding mode in MPEG-4 (Moving Picture Experts Group Phase 4) is described. A video signal comprises two or more video object plains 
(VOP:Video Object Plane). In the case of rectangular shape, VOP is equivalent to the frame and the field in MPEG-1 and 2. Spatial per VOP and it compresses 
with temporal correlation. VOP has a luminance signal and a color-difference signal, and comprises two or more MB(s). MB consists of 16 pixels of every 
direction to a luminance signal. Spatial compression and time compression are performed by this MB unit. A picture is compressed by DCT and quantization in 
spatial compression. A picture is compressed by the motion compensation (MC: Motion Compensation) in time compression. 

0017The compression method of a VOP unit has the formation of a screen inner code (intra coding) coded only by spatial compression, and the coding 
between screens (formation of interchange agreement) coded by spatial compression and time compression. VOP formed into the screen inner code is called 
I-VOP (Intra-VOP), VOP coded between screens has two kinds such as P-VOP (Predictive -VOP) and B-VOP (Bi-directionally predictive-VOP). Only VOP before 
coded in time as the reference VOP is VOP referred to, and P-VOP is VOP which MC of the uni directional was performed and was coded. VOP coded forward 
and backward in time as the reference VOP is VOP referred to, and B-VOP is VOP which bidirectional MC was performed and was coded. In the reference VOP, 
it is VOP which adjoins in time to VOP which should be coded now in VOP decoded in order to have been coded as I-VOP or P-VOP in the past and to use by 
the coding between screens here. At most two VOP(s) are referred to in the coding between of 1 time screens. All MB(s) contained in I-VOP must be coded 
in Intra. On the other hand, each MB contained in P and B-VOP may be coded using which of intra and an interchange. Here, "VOP is formed into a screen 
inner code", "intra coding of the VOP being carried out", and all "it codes by I-VOP" show the same contents. Therefore, a contents difference does not have 
this the expression of any and it means the same contents. 

0018Hereafter, coding processing of MB unit is described briefly. When VOP containing MB which should be coded is I-VOP, the coding part 1 compresses the 
quantized DCT coefficient by variable length coding about a luminance signal and a color-difference signal. The quantized DCT coefficient is computed by DCT 
and quantization being performed about a luminance signal and a color-difference signal. And according to the given order, a bit stream is created with 
header information. 

0019When VOP which contains MB which should be coded on the other hand is except I-VOP, MB on the reference VOP as for which the difference value (MC 
error) in a luminance signal with MB which the coding part 1 should code becomes the smallest is discovered. The reference VOP is coded VOP which adjoins 
in time to VOP containing MB which should be coded. MB which becomes the smallest by MB on the reference VOP in the difference value (MC error) in a 
luminance signal with MB which should be coded is discovered. In order to discover this MB, the motion detection method represented by block matching is 
used. The vector which shows the motion from MB which should be coded to MB to which MC error becomes the smallest is generated. This vector is called a 
motion vector. DCT and quantization are performed to MC error. The obtained motion vector and the DCT coefficient quantized about MC error of the 
luminance signal and the color-difference signal are compressed by variable length coding. The these-compressed motion vector and the quantized DCT 
coefficient are generated as a bit stream according to the given order with header information. 

0020The control signal which specifies the frame rate of the video outputted from the frame rate updating section 4 is outputted to the coding part 1. The 
control signal which specifies the desired value of the code amount generated by coding from the generated code amount control section 5 is outputted to the 
coding part 1. These frame rates and the desired value of a generated code amount are specified, and the coding part 1 performs a frame skip or stuffing. 
And the code sequence by which variable length coding was carried out is outputted to the output buffer 2. 

0021The output buffer 2 outputs the coding row coded in the coding part 1 by the target bit rate. This bit rate is set up by the device control part 7. The 
generated code amount of VOP is computed and the computed generated code amount is outputted to the virtual buffer 3 and the generated code amount 
control section 5. 

0022The initial value (B 0 of drawing 1) of the buffer occupied quantity to which the virtual buffer 3 was set by the device control part 7 is set as buffer 
occupied quantity. The generated code amount (dO of drawing 1) of the first frame obtained from the output buffer 2 is decreased from buffer occupied 
quantity after coding of the first frame (B' 0 of drawing 1). Then, the value which multiplied the time (T of drawing 1: frame interval) to the frame coded next 
by the bit rate is made to increase to buffer occupied quantity (Bj of drawing 1). It performs for every frame decreasing after this the generated code amount 
obtained from the output buffer 2 from buffer occupied quantity, and making the value which multiplied the following frame interval by the bit rate increase 
to buffer occupied quantity. The virtual buffer 3 gives the buffer occupied quantity for every time to the frame rate updating section 4, the generated code 
amount control section 5, and the coding mode set part 6. 

0023The frame rate updating section 4 inspects the buffer occupied quantity after the code amount for one sheet was removed, and outputs the frame rate 
according to the threshold (control point) corresponding to the buffer occupied quantity which performs the frame skip set up beforehand to the coding mode 
set part 6 and the coding part I. The frame rate according to two or more control points is outputted to the coding mode set part 6 and the coding part 1. 
0024The generated code amount control section 5 memorizes the generated code amount of coded VOP which is obtained from the output buffer 2, and the 
average value of the quantizing scale of coded VOP according to coding mode, and gives this to the coding mode set part 6. The VOP coding mode which 
should be coded next is determined in consideration of a predetermined rule (for example, GOP structure of MPEG-2), or change of video mentioned later, 
and this is given to the coding mode set part 6. A quantizing scale, the number of stuffing bits, etc. of VOP which should be coded next are given from the 
buffer occupied quantity of the virtual buffer 3 to the coding part 1. 

0025The coding mode set part 6 presumes the generated code amount at the time of being coded by the coding mode determined by the generated code 
amount control section 5. The buffer occupied quantity of the virtual buffer 3 after coding is presumed from this code amount and the buffer occupied 
quantity obtained from the virtual buffer 3. And the control signal which specifies coding mode suitable for the frame rate corresponding to the presumed 
buffer occupied quantity is outputted to the coding part 1. 

0026The device control part 7 controls the coding part 1, the output buffer 2, the virtual buffer 3, the frame rate updating section 4, and the generated code 
amount control section 5. For example, the bit stream which a device should output is controlled. Specifically, the generated code amount (object bit rate) 
used as a target is directed to the generated code amount control section 5. To suit this object bit rate, the generated code amount control section 5 gives 
various encoding parameters (a quantizing scale, the number of stuffing bits, coding mode, and the number of frame skips) to the coding part 1, and is 
controlling the generated code amount. The frame number per unit time used as a target (target frame rate) is directed to the frame rate updating section 4. 
The value of the amount of initial delay is directed to the virtual buffer 3. The setups of coding modes, such as error resistance, are set as the generated code 
amount control section 5. A control point may be set up. The device control part 7 also performs control of the picture signal inputted into a device. 
0027Drawing 3 is a flow chart which sets up the method to code by presuming the frame rate after coding. 

0028In consideration of improvement in image quality, error tolerance, and random access nature, it may be coded by I-VOP in a bit stream. It is known 
that the code amount which the direction which codes by I-VOP generally generates rather than coding by P-VOP or B-VOP will increase. 
0029Therefore, if I-VOP is inserted frequently, the virtual buffer 3 will become an underflow tendency. When it codes by I-VOP with comparatively many 
generated code amounts by the case where buffer occupied quantity is low, especially (for example, when it is B (min) of drawing 1, drawing 4, or drawing 
5), it is less than B (min), the number of frame skips increases rapidly, and the case where a target frame rate becomes small rapidly may happen. Thus, 
when a user looks at the coded video in which the target frame rate is changed rapidly, appearance senses in many cases that it is bad. So, in changing a 
target frame rate rapidly by insertion of I-VOP, it controls not to code by I-VOP. 

0030A start of coding of video will judge whether intra coding of the VOP is carried out by step ST-A1. The generated code amount control section 5 
performs this judgment. For example, when video changes dramatically (scene change), it controls carrying out intra coding of the VOP etc. When it judges 
with the generated code amount control section 5 carrying out intra coding of the VOP, it progresses to step ST- A3. On the other hand, when it judges with 
the generated code amount control section S not carrying out intra coding of the VOP, it progresses to step ST-A2. 

0031Inter encoding of the VOP is carried out in step ST-A2. Here, it codes to P-VOP. It may be set up code not only to P-VOP but to B-VOP. The generated 
code amount control section 5 may judge, and it may set up code to either P-VOP or B-VOP. It may perform performing about I-VOP in step ST-A3 and ST-A4 
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so that it may mention later also about P-VOP. That is, the frame rate after coding is presumed by P-VOP, and it may be set up code by B-VOP when this 
frame rate is beyond a predetermined value. 

0032In step ST- A3, the frame rate after coding to I-VOP is presumed. And in step ST-A4, it is judged whether the difference of the frame rate presumed by 
step ST- A3 and a desired frame rate is smaller than a predetermined value. That is, it is judged whether the frame rate presumed by step ST-A3 is changing 
rapidly as compared with the frame rate in front of that. In this case, when that difference is smaller than a predetermined value, it will be judged with the 
frame rate not changing rapidly. This predetermined value is given by the device control part 7. When the difference of the frame rate presumed by step ST- 
A4 step ST-A3 and a desired frame rate is smaller than a predetermined value, it progresses to step ST-A5. On the other hand, when the difference of the 
frame rate presumed by step ST-A3 and a desired frame rate is not smaller than a predetermined value, it progresses to step ST-A2. 
0033In order to presume a frame rate, it is necessary to get to know the buffer occupied quantity of the virtual buffer 3 after coding by I-VOP. That is, the 
code amount at the time of coding by I-VOP is needed. Then, the coding mode set part 6 guesses the code amount of I-VOP from the code amount and the 
average value of a quantizing scale of I-VOP coded before. It becomes possible to calculate and guess the buffer occupied quantity of the virtual buffer 3 at 
the time of coding to I-VOP with the code amount of this guessed I-VOP, and the buffer occupied quantity of the virtual buffer 3 in front of coding. A frame 
rate can be presumed if buffer occupied quantity is guessed. 

0034Video is coded by I-VOP step ST- AS. And in order to code the next VOP, it returns to step ST-A1. And the above-mentioned step is repeated until it 
codes all the video. 

0035Drawing 4 is a figure showing an example of the buffer occupied quantity of the virtual buffer 3 to the time in the case where a certain video is coded by 
the conventional method. Drawing 5 is a figure showing the buffer occupied quantity of the virtual buffer 3 to the time at the time of coding the video coded 
as shown in drawing 4 using the video coding equipment in a 1st embodiment. 

0036By performing coding according to the flow chart shown in drawing 3, drawing 5 shows the case where P-VOP coding is carried out by time tj which is 
carrying out intra coding of the VOP by drawing 4 without carrying out intra coding of the VOP. That is, in time tj shown in drawing 4 and drawing 5, although 
it has coded to I-VOP in drawing 4, it has coded to P-VOP by drawing 5. 

0037In drawing 5, as shown in drawing 3, the frame rate after coding to I-VOP is presumed, and when change of the frame rate is less than a predetermined 
value, it is coded by I-VOP (time t'j of drawing 5). If I-VOP coding is carried out by time tj shown in drawing 4, as a frame skip is shown in drawing 4, before 
and after being coded by I-VOP, it will change sharply. In order to avoid changing a frame rate sharply in drawing 5 by the method shown in drawing 3 if 
coded by I-VOP before and behind time tj when coding by time t I# in time t If it codes to P-VOP. 

0038As a result, as shown in drawing 4, change of the number of frame skips which is seen before and behind time t x is tost, and it becomes possible to 
obtain a frame skip with little change. Therefore, it becomes possible to realize improvement in image quality through the whole video. 
0039(A 2nd embodiment) The composition of the video coding equipment concerning a 2nd embodiment of this invention is the same as the composition of 
the dynamic image code-ized device concerning a 1st embodiment. That is, the video coding equipment of this invention consists of the coding part 1, the 
output buffer 2, the virtual buffer 3, the frame rate updating section 4, the generated code amount control section 5, the coding mode set part 6, and the 
device control part 7. However, operations of the frame rate updating section 4 and the generated code amount control section 5 differ. Hereafter, a 2nd 
embodiment is described focusing on this point of difference. 

0040Drawing 6 is a figure showing the buffer occupied quantity of the virtual buffer 3 to the time in a 2nd embodiment of this invention, The history (dashed 
line) of the buffer occupied quantity of the virtual buffer 3 to the time at the time of carrying out three-stage setting out of the threshold (control point) 
corresponding to the buffer occupied quantity which performs a frame skip (Bl, B-2, and B (min) which are shown in drawing 6), It is a figure showing the 
history (solid line) of the buffer occupied quantity of the virtual buffer 3 to the time at the time of accepting and setting up one control point (B shown in 
drawing 6 (min)). 

0041In the example shown in the dashed line of drawing 6, the control point which is the above-mentioned threshold is formed three steps. B (min) is set as 
Bl and the next as B-2 and final full limits as a big value as biggest value. B (min) is a size whose one picture is the grade by which inter encoding was 
carried out, and the difference of Bl and B-2 and the difference of B-2 and B (min) are also the sizes whose one picture is the grade by which inter encoding 
was carried out. However, the manufacturer of video coding equipment is able not to limit these values in particular and to set it as a desired value free. The 
buffer occupied quantity in the virtual buffer 3 is inputted into the frame rate updating section 4, and outputs the target frame rate according to the control 
point set as the three-stage to the coding mode set part 6 and the coding part 1. 

0042Let target frame rates be 15 seconds. That is, in the usual control, the number of sheets which the frame interval to code skips the two whole sheets 
performs coding by setting out of one sheet. The conditions that skip number of sheets when skip number of sheets when the skip number of sheets in 
normal operation is less than 1 and Bl in the frame rate updating section 4 is less than 2 and B-2 skips only skip number of sheets until a buffer is recovered 
when less than 3 and B (min) are inputted. 

0043When buffer occupied quantity comes out enough and it does not set up a control point in the state of the beginning shown in drawing 6 (time is from 0 
to t 3 ) for a certain reason (history of a solid line), the case (history of a dashed line) where a control point is set up shows the history of the same buffer 
occupied quantity. The time interval T fixed from t 0 to t 3 is maintained. This time interval (frame skip) is in inverse proportion to a frame rate. Therefore, 
from t 0 to t 3 of drawing 6, a frame rate saves 15 seconds, and it is coded, skipping one sheet. In time t 3 , it is less than the 1st control point Bl. Therefore, in 
the frame rate updating section 4, the skip number of sheets 2 corresponding to Bl is chosen. Then, the number of frame skips increases. A frame rate will 
become late if it puts in another way. In drawing 6, if time coded next is made into t' 4 , it will be set to t' 4 -t 3 >T. In subsequent V A and t' 5 , t' 6 , skip number of 
sheets continues with 3, 2, and 3, respectively. Thus, in the example of coding shown in drawing 6, when three control points are set up, the number of 
frame skips is not more than three sheets. An opportunity for it to be less than B (min) through the whole decreases remarkably, and stops therefore, 
needing a big frame skip. 

00440n the other hand, when the control point Bl and B-2 are not set up, in time t 3 , skip number of sheets is still 1. And in time t 4 coded next, it is less 
than B (min) in this case. Therefore, in order to prevent buffer underflow, a big frame skip is performed. In the case of drawing 6, it will increase as for skip 
number of sheets more than three sheets. Since the skip number of sheets more than B (min) is always 1, if there is a frame with many code amounts, the 
case where buffer occupied quantity is less than B (min) again may occur frequently like in the case of being time t 6 , As a result, the number of frame 
skips swings between 1 and 3, and change of the number of frame skips becomes large. If it is in this case, even when going back and forth between the 
circumference of the control point of Bl, the change of the number of frame skips does not need to impress a big change in appearance in order to end by 
one sheet. 

0045As mentioned above, when there are two or more control points, as compared with the case where there is only one control point, the size of fluctuation 
of the number of frame skips is small. Therefore, it is not necessary to impress a big change in appearance, since it is set up so that the number of frame 
skips of buffer occupied quantity may change one sheet at a time gradually, as compared with the case where there is only one control point, a possibility 
that the two numbers of frame skips will change suddenly is boiled markedly, and decreases. Since buffer occupied quantity furthermore changes with a high 
value through the whole, making a quantization scale value coarse also decreases, and improvement in image quality through the whole can be realized. 
0046Drawing 7 is a flow chart including the process of setting up the method coded by checking the number of VOP(s) within the past predetermined time by 
which intra coding was carried out in addition to the process of drawing 3. 

0047That by which one step was added to the flow which shows operation of the coding by a 1st embodiment is a flow which shows operation of the coding 
by a 2nd embodiment. That is, a new step is added between step ST-A1 of drawing 3, and step ST- A3. A new step is for making it not frequently coded by I- 
VOP. 

0048As mentioned above, the code amount which the direction which codes by I-VOP generates rather than coding by P-VOP or B-VOP generally increases. 
Therefore, if I-VOP is inserted frequently, the virtual buffer 3 will become an underflow tendency. For example, when video is a scene change which changes 
dramatically, intra coding of the VOP is carried out in many cases, and when a scene change occurs frequently, the virtual buffer 3 becomes an underflow 
tendency easily. Then, it controls not to insert I-VOP frequently. By a 2nd embodiment, the maximum VOP number coded by I-VOP among VOP(s) coded 
within a certain fixed time is restricted as the method. 

0049When it is specifically determined that the generated code amount control section 5 wilt carry out intra coding of the VOP, it progresses to step ST-B3 
from step ST-B1. Step ST-B3 compares the VOP number coded by I-VOP in the past predetermined time with the predetermined number set up beforehand. 
When the VOP number coded by I-VOP is more than a predetermined number, it is considered that buffer occupied quantity is in the state which buffer 
underflow tends to generate small. In this case, it does not code to I-VOP but codes by P-VOP (ST-B2). The set-up predetermined number is usually 
determined depending on the size of the virtual buffer 3. The coding mode set part 6 counts the I-VOP number within the past predetermined time, for 
example, and it is set up so that it may be updated, whenever VOP is coded. 

0050As a result, it can control that I-VOP is frequently inserted by step ST-B3 and the virtual buffer 3 becomes a buffer underflow tendency by it. 
0051AII other steps are the same as that of drawing 3 shown by a 1st embodiment. 
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0052Drawing 8 is a figure showing the buffer occupied quantity of the virtual buffer 3 to the time at the time of setting up the threshold shown in drawing 6, 
using the method shown in the flow chart of drawing 7, and coding video. 

0053When buffer occupied quantity is below in Bl or B-2s, such as time t l# t 2 , and t 3 , as for more than B-2, according to less than B-2 and less than B 
(min), the number of frame skips is increased less than / Bl / and more than B (min) gradually. As a result, in the frame of next doors, the number of 
sheets to skip only has a difference of a maximum of one sheet. By the way, when buffer occupied quantity is less than B (min), the number-of-sheets 
difference skipped between the next frames may be one or more sheets. However, since the number of frame skips is gradually increased by the threshold Bl 
and B-2 more than B (min), it is few when a skip number-of-sheets difference will be two or more sheets. Therefore, it enables a user to see the smoothly 
stable video, without changing a picture sharply for appearance. Since buffer occupied quantity changes with a high value through the whole, making a 
quantization scale value coarse also decreases, and improvement in image quality through the whole can be realized, 

0054The maximum VOP number coded by I-VOP regardless of buffer occupied quantity among VOP(s) coded within a certain fixed time is restricted. When 
coding a certain VOP, it presumes how a frame rate changes and it is determined according to the presumed frame rate whether to carry out intra coding or 
carry out inter encoding. Therefore, it is avoidable that a frame rate falls rapidly. Therefore, it becomes possible to prevent buffer underflow. It can also be 
prevented the number of frame skips increasing suddenly. As a result, it enables a user to see the smoothly stable video, without changing a picture sharply 
for appearance. Since buffer occupied quantity changes with a high value through the whole, making a quantization scale value coarse also decreases, and 
improvement in image quality through the whole can be realized. 

0055This invention is not limited to the embodiment mentioned above, in that technical scope, can change variously and can be carried out. 
0056 

Effect of the Inventions becomes possible to see the smoothly stable video, without changing a picture sharply for appearance, since it is avoidable to 
change the number of frame skips sharply according to the video coding equipment of this invention. 

0057It becomes possible for buffer occupied quantity to make it change with a high value through the whole. Therefore, making a quantization scale value 
coarse also decreases and improvement in image quality through the whole can be realized. 



Field of the InventionThis invention relates to the video coding equipment for elongating the video compressed especially about the video coding 
equipment for compressing video or elongating the compressed video, and outputting a clear picture. 



Description of the Prior ArtWith the video coding equipment in this invention. ITU-T (International.) H.26x and ISO/IEC (International Standards 
Organization/International.) of Telecommunication Union-Telecommunication sector advice It is a device for coding video with the video coding mode 
represented by standard MPEG (Moving Picture Experts Group) for Electrotechnical Commission. That is, video coding equipment performs coding per frame 
using a motion compensation, orthogonal transformation (for example, discrete cosine transform), etc. 

0003Generally, to the video signal inputted, the video coding mode represented by MPEG H.26x of an ITU-T recommendation and standard for ISO/IEC is 
spatial, and applies compression with temporal correlation. Based on the data obtained by this compression, according to the given order, variable length 
coding is performed further, and a code sequence (bit stream) is generated. 

0004Video coding equipment must output the bit stream of the code amount specified according to the predetermined encoding parameter. Furthermore, in 
the buffer by the side of a decoder, a generated code amount must be controlled by the encoder side supposing the occupation of the buffer by the side of a 
decoder so that neither overflow nor underflow arises. This buffer is called a VBV (Video Buffering Verifier) buffer. It may only be called a virtual buffer. As for 
the capacity of the VBV buffer, upper limit was decided by MPEG-4 with the profile and the level. A generated code amount is controlled by the quantizing 
scale used in order to quantize the discrete cosine transform {OCT: Discrete Cosine Transform) coefficient obtained as a result of DCTs being performed by 
every macro block (MB:MacroBlock) in a frame. Generally, a generated code amount and a quantizing scale have a relation of reverse proportion. It is 
possible to change a generated code amount free using this character. 

OOOSHowever, since a quantizing scale generally has restriction, it is impossible to control a generated code amount only by a quantizing scale. Then, when 
there are more generated code amounts than a desired value, the number of frame skips is increased. By increasing the number of frame skips, the frame 
which should be coded can be delayed and the underflow of a VBV buffer can be prevented. Stuffing is performed when there are few generated code 
amounts than a desired value. Stuffing is inserting a redundant bit and prevents overflow of a VBV buffer. 

0006Drawing 1 is a figure showing the conventional method of controlling a virtual buffer, and shows the buffer occupied quantity of the virtual buffer to 
time. As the control method of a frame skip, the boundary value (B (min)) in a VBV buffer is established. The amount of buffers is made to increase by 
skipping until it increases the number of frame skips when less than this (time t 4 shown in drawing 1), and it exceeds a boundary value. 



Effect of the Inventions becomes possible to see the smoothly stable video, without changing a picture sharply for appearance, since it is avoidable to 
change the number of frame skips sharply according to the video coding equipment of this invention. 

0057It becomes possible for buffer occupied quantity to make it change with a high value through the whole. Therefore, making a quantization scale value 
coarse also decreases and improvement in image quality through the whole can be realized. 



Problem(s) to be Solved by the InventionMowever, according to this method, when a state with many generated code amounts continues, a state with 
little buffer occupied quantity continues, and buffer occupied quantity moves the boundary-value circumference up and down. For example, it moves up and 
down like time t 4 shown in drawing 1, and time t 5 . Therefore, the operation which increases the number of frame skips and recovers a buffer, and operation 
of a predetermined frame skip are repeated. As a result, it will be in a state with an unstable frame interval, and quality will worsen dramatically also at 
appearance. 

0008In consideration of improvement in image quality, error tolerance, and random access nature, I-VOP (I-VOP:Intra-Video Object Plane) may be inserted 
into a bit stream. Since I-VOP has comparatively many code amounts, a VBV buffer will become an underflow tendency if I-VOP is inserted frequently. There 
is a problem that the number of frame skips increases rapidly, by inserting I-VOP around said boundary value. 

0009Then, the video where the purpose of this invention raised the quality of the appearance of video and which was stabilized smoothly being reproduced in 
view of the problem in these former is providing the video coding equipment which becomes possible. 



Means for Solving the Problemln video coding equipment which according to this invention performs coding while referring to a virtual buffer for guessing 
an occupation of a buffer by the side of decoding, A generated code amount estimation means which presumes a generated code amount when one picture is 
formed into a screen inner code, A buffer-occupied-quantity estimation means which presumes buffer occupied quantity of said virtual buffer when a 
generated code amount presumed for said one picture is removed, A frame rate estimation means which is based in said presumed buffer occupied quantity, 
and presumes a frame rate, A frame rate setting means which specifies a desired frame rate, and said presumed frame rate, A frame rate judging means 
which judges whether a difference with said frame rate specified is beyond a predetermined value, An encoding control means to control to perform screen 
inner code-ization when a difference of said judged frame rate is less than a predetermined value, and to perform coding by the coding between screens 
when a difference of said judged frame rate is beyond a predetermined value, It is provided by providing video coding equipment. 

OOllThis video coding equipment predicts a virtual buffer after I-VOP coding, and when changing sharply a frame rate obtained from that predicted value 
from the present frame rate, it does not code I-VOP. 

0012Two or more control points are formed in a virtual buffer, and a frame rate updating section which sets up a different frame rate for every point of the is 
provided. This frame rate set up every two or more points is sent to a coding part of video, and a coding part codes by skipping a picture inputted according 
to a specified frame rate. 

0013The number of I-VOP which should be contained within predetermined time from buffer occupied quantity of a virtual buffer is controlled, and buffer 
underflow is controlled. 
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0014 

Embodiment of the InventionHereafter, with reference to Drawings, the embodiment of the video coding equipment of this invention is described. 
(A 1st embodiment) Drawing 2 is a functional block diagram showing the electric internal configuration of the video coding equipment in a 1st embodiment. 
The video coding equipment of this invention consists of the coding part 1, the output buffer 2, the virtual buffer 3, the frame rate updating section 4, the 
generated code amount control section 5, the coding mode set part 6, and the device control part 7. 

OOlSThe coding part 1 inputs the picture signal which should be coded from a picture input device. A picture input device is a signal source of a dynamic 
image signal like a digital video camera or recording playback apparatus. The inputted picture signal is coded according to predetermined video compression 
technology, i.e., MPEG-4 standard method for ISO/IEC and H.263 method of an ITU-T recommendation, or the method which changed these. Coding divides 
each frame contained in the inputted picture signal into a predetermined macro block according to video compression technology. From the coding mode set 
part 6, the coding mode (intra coding or inter encoding) which codes each macro block is inputted. 

0016Below, the video coding mode in MPEG-4 (Moving Picture Experts Group Phase 4) is described. A video signal comprises two or more video object plains 
(VOP:Video Object Plane). In the case of rectangular shape, VOP is equivalent to the frame and the field in MPEG-1 and 2. Spatial per VOP and it compresses 
with temporal correlation. VOP has a luminance signal and a color- difference signal, and comprises two or more MB(s). MB consists of 16 pixels of every 
direction to a luminance signal. Spatial compression and time compression are performed by this MB unit. A picture is compressed by DCT and quantization in 
spatial compression. A picture is compressed by the motion compensation (MC:Motion Compensation) in time compression. 

0017The compression method of a VOP unit has the formation of a screen inner code (intra coding) coded only by spatial compression, and the coding 
between screens (formation of interchange agreement) coded by spatial compression and time compression. VOP formed into the screen inner code is called 
I-VOP (Intra-VOP). VOP coded between screens has two kinds such as P-VOP (Predictive-VOP) and B-VOP (Bi-directionally predictive -VOP). Only VOP before 
coded in time as the reference VOP is VOP referred to, and P-VOP is VOP which MC of the uni directional was performed and was coded. VOP coded forward 
and backward in time as the reference VOP is VOP referred to, and B-VOP is VOP which bidirectional MC was performed and was coded. In the reference VOP, 
it is VOP which adjoins in time to VOP which should be coded now in VOP decoded in order to have been coded as I-VOP or P-VOP in the past and to use by 
the coding between screens here. At most two VOP(s) are referred to in the coding between of 1 time screens. All MB(s) contained in I-VOP must be coded 
in Intra. On the other hand, each MB contained in P and B-VOP may be coded using which of intra and an interchange. Here, "VOP is formed into a screen 
inner code", "intra coding of the VOP being carried out", and all "it codes by I-VOP" show the same contents. Therefore, a contents difference does not have 
this the expression of any and it means the same contents. 

0018Hereafter, coding processing of MB unit is described briefly. When VOP containing MB which should be coded is I-VOP, the coding part 1 compresses the 
quantized DCT coefficient by variable length coding about a luminance signal and a color-difference signal. The quantized DCT coefficient is computed by DCT 
and quantization being performed about a luminance signal and a color-difference signal. And according to the given order, a bit stream is created with 
header information. 

0019When VOP which contains MB which should be coded on the other hand is except I-VOP, MB on the reference VOP as for which the difference value (MC 
error) in a luminance signal with MB which the coding part 1 should code becomes the smallest is discovered. The reference VOP is coded VOP which adjoins 
in time to VOP containing MB which should be coded. MB which becomes the smallest by MB on the reference VOP in the difference value (MC error) in a 
luminance signal with MB which should be coded is discovered. In order to discover this MB, the motion detection method represented by block matching is 
used. The vector which shows the motion from MB which should be coded to MB to which MC error becomes the smallest is generated. This vector is called a 
motion vector. DCT and quantization are performed to MC error. The obtained motion vector and the DCT coefficient quantized about MC error of the 
luminance signal and the color-difference signal are compressed by variable length coding. The these-compressed motion vector and the quantized DCT 
coefficient are generated as a bit stream according to the given order with header information. 

0020The control signal which specifies the frame rate of the video outputted from the frame rate updating section 4 is outputted to the coding part 1. The 
control signal which specifies the desired value of the code amount generated by coding from the generated code amount control section 5 is outputted to the 
coding part 1. These frame rates and the desired value of a generated code amount are specified, and the coding part 1 performs a frame skip or stuffing. 
And the code sequence by which variable length coding was carried out is outputted to the output buffer 2. 

0021The output buffer 2 outputs the coding row coded in the coding part 1 by the target bit rate. This bit rate is set up by the device control part 7. The 
generated code amount of VOP is computed and the computed generated code amount is outputted to the virtual buffer 3 and the generated code amount 
control section 5. 

0022The initial value (B 0 of drawing 1) of the buffer occupied quantity to which the virtual buffer 3 was set by the device control part 7 is set as buffer 
occupied quantity. The generated code amount (dO of drawing 1) of the first frame obtained from the output buffer 2 is decreased from buffer occupied 
quantity after coding of the first frame (B' 0 of drawing 1). Then, the value which multiplied the time (T of drawing 1: frame interval) to the frame coded next 
by the bit rate is made to increase to buffer occupied quantity (B t of drawing 1). It performs for every frame decreasing after this the generated code amount 
obtained from the output buffer 2 from buffer occupied quantity, and making the value which multiplied the following frame interval by the bit rate increase 
to buffer occupied quantity. The virtual buffer 3 gives the buffer occupied quantity for every time to the frame rate updating section 4, the generated code 
amount control section 5, and the coding mode set part 6. 

0023The frame rate updating section 4 inspects the buffer occupied quantity after the code amount for one sheet was removed, and outputs the frame rate 
according to the threshold (control point) corresponding to the buffer occupied quantity which performs the frame skip set up beforehand to the coding mode 
set part 6 and the coding part 1. The frame rate according to two or more control points is outputted to the coding mode set part 6 and the coding part 1. 
0024The generated code amount control section 5 memorizes the generated code amount of coded VOP which is obtained from the output buffer 2, and the 
average value of the quantizing scale of coded VOP according to coding mode, and gives this to the coding mode set part 6. The VOP coding mode which 
should be coded next is determined in consideration of a predetermined rule (for example, GOP structure of MPEG-2), or change of video mentioned later, 
and this is given to the coding mode set part 6. A quantizing scale, the number of stuffing bits, etc. of VOP which should be coded next are given from the 
buffer occupied quantity of the virtual buffer 3 to the coding part 1. 

0025The coding mode set part 6 presumes the generated code amount at the time of being coded by the coding mode determined by the generated code 
amount control section 5. The buffer occupied quantity of the virtual buffer 3 after coding is presumed from this code amount and the buffer occupied 
quantity obtained from the virtual buffer 3. And the control signal which specifies coding mode suitable for the frame rate corresponding to the presumed 
buffer occupied quantity is outputted to the coding part 1. 

0026The device control part 7 controls the coding part 1, the output buffer 2, the virtual buffer 3, the frame rate updating section 4, and the generated code 
amount control section 5. For example, the bit stream which a device should output is controlled. Specifically, the generated code amount (object bit rate) 
used as a target is directed to the generated code amount control section 5. To suit this object bit rate, the generated code amount control section 5 gives 
various encoding parameters (a quantizing scale, the number of stuffing bits, coding mode, and the number of frame skips) to the coding part 1, and is 
controlling the generated code amount. The frame number per unit time used as a target (target frame rate) is directed to the frame rate updating section 4. 
The value of the amount of initial delay is directed to the virtual buffer 3. The setups of coding modes, such as error resistance, are set as the generated code 
amount control section 5. A control point may be set up. The device control part 7 also performs control of the picture signal inputted into a device. 
0027Drawing 3 is a flow chart which sets up the method to code by presuming the frame rate after coding. 

0028In consideration of improvement in image quality, error tolerance, and random access nature, it may be coded by I-VOP in a bit stream. It is known 
that the code amount which the direction which codes by I-VOP generally generates rather than coding by P-VOP or B-VOP will increase. 
0029Therefore, if I-VOP is inserted frequently, the virtual buffer 3 will become an underflow tendency. When it codes by I-VOP with comparatively many 
generated code amounts by the case where buffer occupied quantity is low, especially (for example, when it is B (min) of drawing 1, drawing 4, or drawing 
5), it is less than B (min), the number of frame skips increases rapidly, and the case where a target frame rate becomes small rapidly may happen. Thus, 
when a user looks at the coded video in which the target frame rate is changed rapidly, appearance senses in many cases that it is bad. So, in changing a 
target frame rate rapidly by insertion of I-VOP, it controls not to code by I-VOP. 

0030A start of coding of video will judge whether intra coding of the VOP is carried out by step ST-A1. The generated code amount control section 5 
performs this judgment. For example, when video changes dramatically (scene change), it controls carrying out intra coding of the VOP etc. When it judges 
with the generated code amount control section 5 carrying out intra coding of the VOP, it progresses to step ST- A3. On the other hand, when it judges with 
the generated code amount control section 5 not carrying out intra coding of the VOP, it progresses to step ST-A2. 

0031Inter encoding of the VOP is carried out in step ST-A2. Here, it codes to P-VOP. It may be set up code not only to P-VOP but to B-VOP. The generated 
code amount control section 5 may judge, and it may set up code to either P-VOP or B-VOP. It may perform performing about I-VOP in step ST-A3 and ST-A4 
so that it may mention later also about P-VOP. That is, the frame rate after coding is presumed by P-VOP, and it may be set up code by B-VOP when this 
frame rate is beyond a predetermined value. 

0032ln step ST- A3, the frame rate after coding to I-VOP is presumed. And in step ST-A4, it is judged whether the difference of the frame rate presumed by 
step ST- A3 and a desired frame rate is smaller than a predetermined value. That is, it is judged whether the frame rate presumed by step ST- A3 is changing 
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rapidly as compared with the frame rate in front of that. In this case, when that difference is smaller than a predetermined value, it will be judged with the 
frame rate not changing rapidly. This predetermined value is given by the device control part 7. When the difference of the frame rate presumed by step ST- 
A4 step ST- A3 and a desired frame rate is smaller than a predetermined value, it progresses to step ST-A5. On the other hand, when the difference of the 
frame rate presumed by step ST-A3 and a desired frame rate is not smaller than a predetermined value, it progresses to step ST-A2. 
0033In order to presume a frame rate, it is necessary to get to know the buffer occupied quantity of the virtual buffer 3 after coding by I-VOP. That is, the 
code amount at the time of coding by I-VOP is needed. Then, the coding mode set part 6 guesses the code amount of I-VOP from the code amount and the 
average value of a quantizing scale of I-VOP coded before. It becomes possible to calculate and guess the buffer occupied quantity of the virtual buffer 3 at 
the time of coding to I-VOP with the code amount of this guessed I-VOP, and the buffer occupied quantity of the virtual buffer 3 in front of coding. A frame 
rate can be presumed if buffer occupied quantity is guessed. 

0034Video is coded by I-VOP step ST-A5. And in order to code the next VOP, it returns to step ST-A1. And the above-mentioned step is repeated until it 
codes all the video. 

0035Drawing 4 is a figure showing an example of the buffer occupied quantity of the virtual buffer 3 to the time in the case where a certain video is coded by 
the conventional method. Drawing 5 is a figure showing the buffer occupied quantity of the virtual buffer 3 to the time at the time of coding the video coded 
as shown in drawing 4 using the video coding equipment in a 1st embodiment. 

0036By performing coding according to the flow chart shown in drawing 3, drawing 5 shows the case where P-VOP coding is carried out by time t } which is 
carrying out intra coding of the VOP by drawing 4 without carrying out intra coding of the VOP. That is, in time tj shown in drawing 4 and drawing 5, although 
it has coded to I-VOP in drawing 4, it has coded to P-VOP by drawing 5. 

0037In drawing 5, as shown in drawing 3, the frame rate after coding to I-VOP is presumed, and when change of the frame rate is less than a predetermined 
value, it is coded by I-VOP (time t'j of drawing 5). If I-VOP coding is carried out by time tj shown in drawing 4, as a frame skip is shown in drawing 4, before 
and after being coded by I-VOP, it will change sharply. In order to avoid changing a frame rate sharply in drawing 5 by the method shown in drawing 3 if 
coded by I-VOP before and behind time tj when coding by time t v in time tj, it codes to P-VOP. 

0038As a result, as shown in drawing 4, change of the number of frame skips which is seen before and behind time tj is lost, and it becomes possible to 
obtain a frame skip with little change. Therefore, it becomes possible to realize improvement in image quality through the whole video. 
0039(A 2nd embodiment) The composition of the video coding equipment concerning a 2nd embodiment of this invention is the same as the composition of 
the dynamic image code-ized device concerning a 1st embodiment. That is, the video coding equipment of this invention consists of the coding part 1, the 
output buffer 2, the virtual buffer 3, the frame rate updating section 4, the generated code amount control section 5, the coding mode set part 6, and the 
device control part 7. However, operations of the frame rate updating section 4 and the generated code amount control section 5 differ. Hereafter, a 2nd 
embodiment is described focusing on this point of difference. 

0040Drawing 6 is a figure showing the buffer occupied quantity of the virtual buffer 3 to the time in a 2nd embodiment of this invention, The history (dashed 
line) of the buffer occupied quantity of the virtual buffer 3 to the time at the time of carrying out three-stage setting out of the threshold (control point) 
corresponding to the buffer occupied quantity which performs a frame skip (Bl, B-2, and B (min) which are shown in drawing 6), It is a figure showing the 
history (solid line) of the buffer occupied quantity of the virtual buffer 3 to the time at the time of accepting and setting up one control point (B shown in 
drawing 6 (min)). 

0041In the example shown in the dashed line of drawing 6, the control point which is the above-mentioned threshold is formed three steps. B (min) is set as 
Bl and the next as B-2 and final full limits as a big value as biggest value. B (min) is a size whose one picture is the grade by which inter encoding was 
carried out, and the difference of Bl and B-2 and the difference of B-2 and B (min) are also the sizes whose one picture is the grade by which inter encoding 
was carried out. However, the manufacturer of video coding equipment is able not to limit these values in particular and to set it as a desired value free. The 
buffer occupied quantity in the virtual buffer 3 is inputted into the frame rate updating section 4, and outputs the target frame rate according to the control 
point set as the three-stage to the coding mode set part 6 and the coding part 1. 

0042Let target frame rates be 15 seconds. That is, in the usual control, the number of sheets which the frame interval to code skips the two whole sheets 
performs coding by setting out of one sheet. The conditions that skip number of sheets when skip number of sheets when the skip number of sheets in 
normal operation is less than 1 and Bl in the frame rate updating section 4 is less than 2 and B-2 skips only skip number of sheets until a buffer is recovered 
when less than 3 and B (min) are inputted. 

0043When buffer occupied quantity comes out enough and it does not set up a control point in the state of the beginning shown in drawing 6 (time is from 0 
to t 3 ) for a certain reason (history of a solid line), the case (history of a dashed line) where a control point is set up shows the history of the same buffer 
occupied quantity. The time interval T fixed from t 0 to t 3 is maintained. This time interval (frame skip) is in inverse proportion to a frame rate. Therefore, 
from t 0 to t 3 of drawing 6, a frame rate saves 15 seconds, and it is coded, skipping one sheet. In time t 3 , it is less than the 1st control point Bl. Therefore, in 
the frame rate updating section 4, the skip number of sheets 2 corresponding to Bl is chosen. Then, the number of frame skips increases. A frame rate will 
become late if it puts in another way. In drawing 6, if time coded next is made into t' 4 , it will be set to t' 4 -t 3 >T. In subsequent t' 4 and t' 5 , t' 6 , skip number of 
sheets continues with 3, 2, and 3, respectively. Thus, in the example of coding shown in drawing 6, when three control points are set up, the number of 
frame skips is not more than three sheets. An opportunity for it to be less than B (min) through the whole decreases remarkably, and stops therefore, 
needing a big frame skip. 

00440n the other hand, when the control point Bl and B-2 are not set up, in time t 3 , skip number of sheets is still 1. And in time t 4 coded next, it is less 
than B (min) in this case. Therefore, in order to prevent buffer underflow, a big frame skip is performed. In the case of drawing 6, it will increase as for skip 
number of sheets more than three sheets. Since the skip number of sheets more than B (min) is always 1, if there is a frame with many code amounts, the 
case where buffer occupied quantity is less than B (min) again may occur frequently like in the case of being time t 6 . As a result, the number of frame 
skips swings between 1 and 3, and change of the number of frame skips becomes large. If it is in this case, even when going back and forth between the 
circumference of the control point of Bl, the change of the number of frame skips does not need to impress a big change in appearance in order to end by 
one sheet. 

0045As mentioned above, when there are two or more control points, as compared with the case where there is only one control point, the size of fluctuation 
of the number of frame skips is small. Therefore, it is not necessary to impress a big change in appearance, since it is set up so that the number of frame 
skips of buffer occupied quantity may change one sheet at a time gradually, as compared with the case where there is only one control point, a possibility 
that the two numbers of frame skips will change suddenly is boiled markedly, and decreases. Since buffer occupied quantity furthermore changes with a high 
value through the whole, making a quantization scale value coarse also decreases, and improvement in image quality through the whole can be realized. 
0046Drawing 7 is a flow chart including the process of setting up the method coded by checking the number of VOP(s) within the past predetermined time by 
which intra coding was carried out in addition to the process of drawing 3. 

0047That by which one step was added to the flow which shows operation of the coding by a 1st embodiment is a flow which shows operation of the coding 
by a 2nd embodiment. That is, a new step is added between step ST-A1 of drawing 3, and step ST- A3. A new step is for making it not frequently coded by I- 
VOP. 

0048As mentioned above, the code amount which the direction which codes by I-VOP generates rather than coding by P-VOP or B-VOP generally increases. 
Therefore, if I-VOP is inserted frequently, the virtual buffer 3 will become an underflow tendency. For example, when video is a scene change which changes 
dramatically, intra coding of the VOP is carried out in many cases, and when a scene change occurs frequently, the virtual buffer 3 becomes an underflow 
tendency easily. Then, it controls not to insert I-VOP frequently. By a 2nd embodiment, the maximum VOP number coded by I-VOP among VOP(s) coded 
within a certain fixed time is restricted as the method. 

0049When it is specifically determined that the generated code amount control section 5 will carry out intra coding of the VOP, it progresses to step ST-B3 
from step ST-B1. Step ST-B3 compares the VOP number coded by I-VOP in the past predetermined time with the predetermined number set up beforehand. 
When the VOP number coded by I-VOP is more than a predetermined number, it is considered that buffer occupied quantity is in the state which buffer 
underflow tends to generate small. In this case, it does not code to I-VOP but codes by P-VOP (ST-B2). The set-up predetermined number is usually 
determined depending on the size of the virtual buffer 3. The coding mode set part 6 counts the I-VOP number within the past predetermined time, for 
example, and it is set up so that it may be updated, whenever VOP is coded. 

0050As a result, it can control that I-VOP is frequently inserted by step ST-B3 and the virtual buffer 3 becomes a buffer underflow tendency by it. 
0051AII other steps are the same as that of drawing 3 shown by a 1st embodiment. 

0052Drawing 8 is a figure showing the buffer occupied quantity of the virtual buffer 3 to the time at the time of setting up the threshold shown in drawing 6, 
using the method shown in the flow chart of drawing 7, and coding video. 

0053When buffer occupied quantity is below in Bl or B-2s, such as time t v t 2 , and t 3 , as for more than B-2, according to less than B-2 and less than B 
(min), the number of frame skips is increased less than / Bl / and more than B (min) gradually. As a result, in the frame of next doors, the number of 
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sheets to skip only has a difference of a maximum of one sheet. By the way, when buffer occupied quantity is less than B (min), the number-of-sheets 
difference skipped between the next frames may be one or more sheets. However, since the number of frame skips is gradually increased by the threshold Bl 
and B-2 more than B (min), it is few when a skip number-of-sheets difference will be two or more sheets. Therefore, it enables a user to see the smoothly 
stable video, without changing a picture sharply for appearance. Since buffer occupied quantity changes with a high value through the whole, making a 
quantization scale value coarse also decreases, and improvement in image quality through the whole can be realized. 

0054The maximum VOP number coded by I-VOP regardless of buffer occupied quantity among VOP(s) coded within a certain fixed time is restricted. When 
coding a certain VOP, it presumes how a frame rate changes and it is determined according to the presumed frame rate whether to carry out intra coding or 
carry out inter encoding. Therefore, it is avoidable that a frame rate falls rapidly. Therefore, it becomes possible to prevent buffer underflow. It can also be 
prevented the number of frame skips increasing suddenly. As a result, it enables a user to see the smoothly stable video, without changing a picture sharply 
for appearance. Since buffer occupied quantity changes with a high value through the whole, making a quantization scale value coarse also decreases, and 
improvement in image quality through the whole can be realized. 

0055This invention is not limited to the embodiment mentioned above, in that technical scope, can change variously and can be carried out. 



Brief Description of the Drawings 

Drawing lit is a figure showing the conventional method of controlling a virtual buffer, and is a figure showing the buffer occupied quantity of the virtual 
buffer to time. 

Drawing 2It is a functional block diagram showing the electric internal configuration of the video coding equipment in the embodiment of this invention. 
Drawing 3It is a flow chart showing the procedure for setting up the method in a 1st embodiment of this invention to code, and is a flow chart which sets up 
the method to code by presuming the frame rate after coding. 

Drawing 4It is a figure showing an example of the buffer occupied quantity of the virtual buffer to the time in the case where a certain video is coded by the 
conventional method. 

Drawing 5It is a figure showing the buffer occupied quantity of the virtual buffer to the time at the time of coding the video coded as shown in drawing 4 
using the video coding equipment in a 1st embodiment of this invention. 

Drawing 6It is a figure showing the buffer occupied quantity of the virtual buffer to the time in a 2nd embodiment of this invention, The history (dashed line) 
of the buffer occupied quantity of the virtual buffer to the time at the time of carrying out three-stage setting out of the threshold (control point) 
corresponding to the buffer occupied quantity which performs a frame skip, It is a figure showing the history (solid line) of the buffer occupied quantity of the 
virtual buffer to the time at the time of accepting and setting up one control point. 

Drawing 7It is a flow chart showing the procedure for setting up the method in a 2nd embodiment of this invention to code, and is a flow chart including the 
process of setting up the method coded by checking the number of VOP(s) within the past predetermined time by which intra coding was carried out in 
addition to the process of drawing 3. 

Drawing 8It is a figure showing the buffer occupied quantity of the virtual buffer to the time at the time of setting up the threshold shown in drawing 6, 
using the method shown in the flow chart of drawing 7, and coding video. 
Description of Notations 

1 Coding part 

2 Output buffer 

3 Virtual buffer 

4 Frame rate updating section 

5 Generated code amount control section 

6 Coding mode set part 

7 Device control part 



Drawing 1 

For drawings please refer to the original document. 



Drawing 2 

For drawings please refer to the original document. 



Drawing 4 

For drawings please refer to the original document. 



Drawing 5 

For drawings please refer to the original document. 



Drawing 3 

For drawings please refer to the original document. 



Drawing 6 

For drawings please refer to the original document. 



Drawing 7 

For drawings please refer to the original document. 



Drawing 8 

For drawings please refer to the original document. 



For drawings please refer to the original document. 
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[003 3] 7k-Ak-h^Jt^-fStdi, I-VO 
P-CBr^kflWHBa^^y 7 r 3«A 7 7 r £Wft£*D6 
iftgtf&l.. I-VOPtffHtLft«^<0 

opm^At^&^Ht^^-^co^mt af 
cioidsiisit^i - vop^it, flrt-ftEfroiR 

«7^y7r 3C7)A'.y7Tt5^*ttJ; , y, I "VOPtC 

^•ftL^^O(RS^"-y 7 r 3 W^y 7 7 i^MSrfl- 
*L"C«a-r4 C blMffllztch . ^*>y 7 r £1r«tfflt 

[0 0 34] 7.x-y7°ST-A5-C{i, I-VOPTf 

WK&WHk**. zix. oi'wopjfffm 

Xf 7 7ST-AlCI5. ZLX. £XCT>m 

[0035] @4{i. wm^zi^xhmmm 

•y7rA : fi'*c 7 )-0<J5:*-ril7:'J>l). 05ii. 134 
i d MPHtZtillMBt:. %> 1 WWBBWfc 

BtrafcWI){5»7N'-y 7 r 3W<7 7 r iWM^^-TH 

[0036] H3fc*Lfc7o-0fcLfc#-5Tfir#*l: 
SrHtftSifctiO. H4-CVOPSr-fyh7^k 
LTV»&H*3Hlt i X\ VOPZ4yYy®mtL%^X 
P-VOP«=^fl:tfe«^*BI5(i*LTV^. 
*>. H4tJj;l^5^SitTV^^ti tcJsv^T. 



(6) P12 003-92759 (P2003-92759A) 



H4TJ4. I -VOPlZftmtlX^ltiK 
P-VOPfcar*fcl/CV>*. 
[00 37] 05^4, 03fcjj*W:J:5fcl -VOP 

-A^-h^bw»ft«iairt , r**«*fcii» i -v 

*l*l*»t, -ei-VOPflr-^kSil&i;. 71^- AX 

* >y r **H4 Ci^ioc, i - v o p fcflpf-fts 

f£T"(4 I - V O P fcflr^fcS ft* i: 7 P-A h 
&<£»LTL*3£i:£aHtSfcttfcl**!lti T"(4P 

-voptcwutt*. 

[0038] H4 4 0 £Sm 1 1 

[0039] (m2<7)nm&m) z<D%w<7)&2com& 

7r2, |Rg/<77T3, 7P-Alx-bM«TgP4. % 

»J««7*»fe*4. tot'U 7l/-AU-He§r»4tJ 
4l/^^*$iJfflia55Wii^M 3 S:«». OTs -coffl 
g££ «W>t LTm 2 ^HIBgJB* KB^4 . 

[0040] 116(4. z<7)%,wcr>m2(?>m&Bmi,zm 

ltirmzm-&fo&'<v 7 r 3W<» 7 r £Wft£^-t 

mznmh l * ( wmm y h £ 3 ae^s t 

(06fc*3*VO^Bl, B2.fc4t/B (m 
in)) wmfctttUBlBS* y 7 7 3 OA y 7 y £W 
JtoJKE ( B8K ) . *9»*>f y h £ 1 m*B5£ Lfz 
(06fc^$ilT^&B (mi n) ) 0)B$filK*ft 
&fi^N'v7r3^v7yA^r*OJSK (UK) fc£ 

[004 1] 06^«e»t*S<lTV^0l"C«. ±IB60 

l # v ^mx-bmmw y^m mmim^Kx ^ a . 

LTBl, o8ffc*S*tfl: LTB2. 
LX B (man) aTO&hW*. 
B (rain)(4 1 MR*M X^-«fft§ftfcg«« 
9. B ltB2t<7m. #4tfB2£B (m 

i n ) k<7)mi> i lit&aM y ?-$fttt.Ztitzmm<7Xk 

.1 i: mmX'h h . {RJS^n* 7773 77r 
£W*(4. 7U-AU-Mgfra54(CA7J$<l. 3gPg 

izmMztitzfflWM ynzmttzmyu-Au-h 



[0042] £tz. BmyU-Als-hZtikltf. ft 
1 5ftfc-T&. muMfflmt. fftttth 

7W-ABIHS*«2tScr fc^-y 7^-4«3R{4 1 fti 

SffiWT'COX* 77Mli 1, Bl £TB 
X*77tfc&l42. B2£T0o£^£<7)7>*-y7m 
(43. B (mi n) £TdH*:i§-£(4^7 7 rtflllfg-t 
6 1 X<F>X* 7 TMfc'WJf 7 rth t V > 3 
JiZtlX^h. 

[0043] m6lzmtlX^Z>m)nW& {VffltfO 

frt>t 3 tx) x-it. ^v7T&^mtf-tftX'hz>tdh 
®mm yhzmfeiK^miz (im^ss) . tutx 
mm#4 y h iw&t&Mfe mmmm) t uzm 

7 r £#*<7)JS112:^-t. t o frt> 1 3 £T\ - 

^mmmT^^x^h. znmmm® (7u-a 

^77) (4. 7U-A^-hfcRJt0!LTV*S. I9>i 
(3, 06<Ot o *»tt 3 *-C'71^-AP-h(±. ftl 5 

s. mmt 3 itz&ux. mmmmm yhBizr® 

l»SXJf77lK2«$h^ -T4t, 7P 
-A7>df vTWPIHtl. fUSthJli. 7l/-AU-b 

4 fcfSt. t" 4 -t 3 >T(;^r4. >ewa<o 

t ■ 4 . t • 5 . t ■ 6 Hi, ZtlZ'tlX* 7 7TO* J 

3.2, 3tm<. ;»iac, me^itz^itm 

^77111 3ft4 9£<3ro-0^\ Lfe^T. 
^ft&jitTB (mi n) ^THll»^* { l : L<iiii'- 
U A#^7U-AXdf 7 7S:^fcL^<^S. 
[0044] -7f . IWftl^ yhBl t$iX/B2tf3B. 

ztix^K^msu, m\t 3 iz&^x. z^'vznm 
tow**?hi. zix. zcom^te, orient 

-t&V$Mt 4 ttJl^TB (mi n) SrT@4. «9>t(c:. 
^7 7r7y/-7D-*»Otftl:, 7C#^7U-A 
7dr 7 7^'^T§il4. H60^{4. x#.y7tS«Ji 
3|5c4 0^< ; Sr-oi:L^a. £tz. B (mi n ) &±<7) 

x*vT&m*fti<z\xbhcox\ fftmn&wu- 

StfB (m i n) SrTEloTt* 3 *&^JS« L 7 S . 
•e^m> 7P—AX*.y7»'l t 3<Dffl£«pt>%\ 

&X'$>tm. b i nmwiM ybwfljasRofc 

O-T^^T'i. 7 l^-A7^f 7 7m>3Ei!j(4 1 IXT't 

[0045] W±^4 3 fc, «0«!#>f y 
-^■(4. !H»^>fyh*«loU*^v^fc:Jt«Lr. 7 

-y 7^C<0(9> 6 #'C07C# $ V ^ . Lfctfo 

r AWS^7l/-AX^ 7 71$*^ l«rt*oRltWlc«ft 
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^m^mm ixm Mt-e»r & t^&^it^ 
[0046] mm, m3<7)Tmzmix , 

t lz£ ^Xft^fc^h1y£*%mthJMZtXs 7 o- 

[ o o 4 7 ] n i <7)mimmx'<7)tt^it<7)mftz*t7 

X'<»'ftmt<7)MftZ^-t7u-xh& . f&*>^ia3tf> 
Xf'//ST-Al fc7f 7 7°ST-A3fcC7)^fT?t 

-voPt^CS^uj: o iz-fhtzfrco^xh 

[0 048] IMUziolZ^ H&W£P-VOP^B 
-VOPTl?^k5:ffdJ:0i, I-VOPmf^ 

Iz I — VOP £if A^S i: , {R«a* y 7 7 3 &7yf- 

LTm20HSfi^T*{i, fci-JIWBIrt-CfiFtfl:* 
JlfcVOPO^^ I-VOPTflr^k$<l&ft*<0V 

[0049] mmzte, %tffMMIN 5 #V O P 

-Bl*^Xx>yrST-B3{;jttf. Xf7 7*ST- 
B 3 t'fi. iteOBf^iiSiartfc I -VO PlcflHHfcSft. 

fcvopa»ft^3&>tftasL"r*s0f^Ri:JtiS[-r 

6. I-VOPtflr^S<lfcVOP»*«m®RlJUi-C 
A'-y 7 r < A*. y 7 7 T^r 

tt. i-voPKflFWH\ p-vop-cflHHrf* 

(ST-B2) . HkfelX$>m%ffli. »#, 

fl^y7r3<0*&§fctt#LT8a£$fi4. 3i*<7) 

m«B#^i*i^ i - v o pmt . rc 1 i im^it*- m 

\Z7 >y 7f- h $*U> 4 d fcRjg LX& < . 
[0050] dcO^H, Xf77ST-B3CJ:ot, 
I -VOP&mmzW A£*U&®Ay 7 r 3**A>y 7 r 

Ty^-7oH«l*Ifc:*4Ci:S:«iiSi|-tftii:36*T# 

I. 

[005 1 ] f$*K9*f 7711 "f^-caSl <0|tatJKB 
[0 0 5 2] 08ti. H6(c*SftfcL*vHlMBfcl, 



v7T&Gmz*-tmx'$>&. 

[0053] A'-^rAti^t, „ t 2 . t 3 Sf 
OBl*fc{iB2WTt*S«^«, B21il±Bl* 
i.B(min) JjLhB 2*p. B (m i n ) *?S(Ct 
fctfof . SHWfc7 U-AT.* v7Wfi£ < Ztll . 

-e«*sa. Kra±o7^-At«. ^^'//tsMi: 

WJttfB (min)*l^(i, m<r>7U-&tnffl 
X'Z* >y 7"r&ft&g# 1 tSU2Lh-Cfc& "Iffitttt** . 
L*»U B (mi n) JJLbOLS^ffiB l*i JtfB 2"C 
KPgW£7P-AX* •y7»'tiiD§^-CV^^T'\ X 
* -y 7lJ«ife&«2WSLh(c44*-&t*JwarV^ Lfctfo 
T . a— «f (4 . Hfc g (cfffttf* S < S3ft* hZbtt 
< . 7. A-Xt£5g I feliKAi -I i: #*r«gfc& 
4. 4fc. A'-y7 7^*^ft$r3itTiiV^T« 

0, £mMLtimmft±.t:mix'Z&. 

[0 0 54] ttz. A' 7 7r^i(:iH«:, *S- 
J&WrtTflHWfcStifcVO PO a I - VO PTflF 
-t^iiSSA^vop^^JRgLtv^. s^fc, $, 
4VOP*«^-r*l6t. 7U-AW-- h^fc'd^k 

4 yh7®^t*-r&fr4 y?-&mtz-f 

rry^-7Q-SrR6<'Cli:*«irffife:*S. 71/ 

4fe, A*-y7 7(5*ft*^«s€:iiL-C^V^'C«l 
[0055] ^cA%Hj(i^ ±JfiLfclOS^)»Bfc:|R€S 
[0056] 

7 P-A7.dr >y 7fiC#A# < S»"t £ CI t *»»t4 i fc 
4. 

[005 7] 4^, A'.y7r A%a*^ft«raurs^ 

mxim zit&zt tf*imi,z% h „ u^t, tut 

[lafficrjfgjji&fra] 

inn isa-7 7 7 *wm-&m&>imt:wfw? 
h->x , B#iatc*tts{s«7N'«y 7 t^<7 7 7 A^a^ 



(8) HB2 003-92759 (P2003-92759A) 



<n%%mnw$&L*^®m7* y ? mx'$>&. 
[04 ] wmumz^x hhwmm^itthm 

£?co . ^HSI^-rSiSM^ 7 7 7 WW 7 T a* 

[05] 04 {^^ixsiot^^ in. mmmz. 

Wy 7 r £3r:I£*-f 0"C$>& . 

[06] L<r>%mm2<r>mrnm^a»muzm 

hW&^v 7 r W-y 7 r 0TS>-?-C . 7 

£VMI ($iJW^yh) j3SPglSSt^^B#^ 
(KS^ •/ 7 r W >y 7 r A^TiOlS® ( WM ) 



^{SSA'.y7r^N'>y7r.5**^H)S (9liS) i:5r* 
-f0T&l>. 

[07] zc7)m^m2^mmmmi / zmin^it-ti 

3 outran*. T . i§*<DJ3f5£i$llf*l?M y h 

&*WLfc$hJLm:%ts7u-mx'foh. 

[08] 06£j*$*ifcL#V^£|&£L"C, 07O7 

B$ISfc*ft&<Kg7'\'<y 7 rCOA'.y 7 7 
*-f0T"S>&. 
[flF^tBBJJ] 
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7oy h^-'J^WiZ 



(72)&BM Jim $fr F?-A(##) 5C059 KK01 LB07 MAOO MA05 MA14 

W£)\\m)\\^mmWmom& ffc££tt MA23 MCll MC38 MEOl NN21 

3K2««PW»5rrt PP05 PP06 PP07 TA07 TA18 

TB07 TC16 TC19 TC20 TC27 
TC38 TD12 UA02 UA32 UA38 
5J064 BB01 BC01 BC26 BD03 



